Experimental Approaches to the Study of Personality

A review of the use of experimental techniques to develop and test theories of personality processes. Threats to valid inference including problems of scaling, reliability, and unintended confounds are considered. Basic experimental designs are discussed as ways of eliminating some, but not all threats to validity. A number of basic analytical procedures are demonstrated using simulated data that can be accessed from the web based appendix. Personality is an abstraction used to explain consistency and coherency in an individuals pattern of affects, cognitions, desires and behaviors. What one feels, thinks, wants and does changes from moment to moment and from situation to situation but shows a patterning across situations and over time that may be used to recognize, describe and even to understand a person. The task of the personality researcher is to identify the consistencies and differences within and between individuals (what one feels, thinks, wants and does) and eventually to try to explain them in terms of set of testable hypotheses (why one feels, thinks, wants and does). Personality research is the last refuge of the generalist in psychology: it requires a familiarity with the mathematics of personality measurement, an understanding of genetic mechanisms and physiological systems as they interact with environmental influences to lead to development over the life span, an appreciation of how to measure and manipulate affect and cognitive states, and an ability to integrate all of this into a coherent description of normal and abnormal behavior across situations and across time. Although the study of personality is normally associated with correlational techniques relating responses or observations in one situation or at one time with responses in other situations and other times, it is also possible to examine causal relations through To appear in Personality Research Methods B. Robins, C. Fraley and R. Krueger, eds Guilford, 2007 address comments to [email protected] EXPERIMENTAL APPROACHES 2 the use of experimental methods. This chapter will outline some of the challenges facing personality researchers and suggest that an experimental approach can be combined with more traditional observational techniques to tease out the causal structure of personality. Central to our analysis is the distinction between personality traits and personality states. Experimental studies do not change trait values, but rather combine (and perhaps interact) with traits to affect the current state. States can be thought of as the current values of ones affects, behaviors, cognitions and desires while traits have been conceptualized as average values of these states or alternatively the rates of change in these states (Ortony, Norman, and Revelle, 2005). In more cognitive terms, traits are measures of chronic accessibility or activation, and states are levels of current activation. (Although many personality theorists do not include intellectual ability in their theories, those of us who do consider ability traits as reflecting maximal performance while non-cognitive traits of personality reflect typical or average behavior.) It is perhaps useful here to think analogically and to equate states with todays weather and traits as long terms characteristics of weather, that is to say, climate. On any particular day, the weather in a particular location can be hot or cold, rainy or dry. But to describe the climate for that location is more complicated, for it includes among other aspects a description of the seasonal variation in temperature and the long term likelihood of draught, blizzards or hurricanes. Extending this analogy, climatologists explain differences in climate between locations in terms of variations in solar flux associated with latitude and proximity to large bodies of water, and changes in climate in terms of long term trends in e.g., greenhouse gases in the atmosphere. The role of the personality researcher is analogous to the meteorologist and climatologist, trying to predict someones immediate states as well as understanding and explaining long term trends in feelings, thoughts and actions. Integrating Two Alternative Research Approaches Psychological research has traditionally been described in terms of two contrasting approaches: the correlational versus the experimental (viz., the influential papers by Cronbach, 1957, 1975; and Eysenck, 1966, 1997). Francis Galton and his associate Karl Pearson introduced the correlation coefficient as an index of how individual differences on one variable (e.g., the height of ones parents or ones occupation) could be related to individual differences in another variable (e.g., ones own height or to ones reaction time). Correlational approaches have been used in personality research since Galton to predict a multitude of outcomes (e.g., adjustment, career choice, health, leadership effectiveness, marital satisfaction, romantic preferences, school achievement, and job performance) and when combined with known family structures (e.g., parents and their offspring, monozygotic or dizygotic twins with each other, adopted and biological siblings) and structural equation models have allowed for an examination of the genetic basis of personality. Applying structural techniques such as factor analysis to covariance or correlation matrices of self and other descriptions has led to taxonomic solutions such as the Giant Three or Big EXPERIMENTAL APPROACHES 3 Five trait dimensions. The emphasis in correlational research is on variability, correlation, and individual differences. Central tendencies are not important; variances and covariances are. The primary use of correlational research is in describing how people differ and how these differences relate to other differences. Unfortunately for theoretical inference, that two variables are correlated does not allow one to infer causality. (E.g., that foot size and verbal skill are highly correlated among preteens does not imply that large feet lead to better verbal skills, for a third variable, age, is causally related to both. Similarly, that yellowed fingers, yellowed teeth and bad breath are associated with subsequent lung cancer should not lead to a run on breath fresheners or gloves, but rather to stopping smoking.) A seemingly very different approach to research meant to tease out causality is the use of experimental manipulation. The psychological experiment, introduced by Wundt and then used by his students and intellectual descendants allows one to examine how an experimental manipulation (an Independent Variable) affects some psychological observation (the Dependent Variable) which, in turn, is thought to represent a psychological construct of interest. The emphasis is upon central tendencies, not variation, and indeed, variability not associated with an experimental manipulation is seen as a source of noise or error that needs to be controlled. Differences of means resulting from different experimental conditions are thought to reflect the direct causal effects of the IV upon the DV. Threats to the validity of an experiment may be due to confounding the experimental manipulation with multiple variables or poor definition of the dependent variables or an incorrect association between observation and construct. One reason that correlational and experimental approaches are seen as so different is that they have traditionally employed different methods of statistical analysis. The standard individual differences/correlational study reports either a regression weight or a correlation coefficient. Regression weights are measures of how much does variable Y change as a function of a unit change in variable X. Correlations are regressions based upon standard scores, or alternatively the geometric mean of two regression slopes (X upon Y and Y upon X). A correlation is an index of how many standardized units does Y change for a standardized unit of X. (By converting the raw Y scores into standardized scores, zy = (Y −Y.)/s.d.Y , one removes mean level as well as the units of measurement of Y.) Experimental results, on the other hand, are reported as the differences of the means of two or more groups, with respect to the amount of error within each group. Students t-test and Fishers F test are the classic way of reporting experimental results. Both t and F are also unit free, in that they are functions of the effect size (differences in means expressed in units of the within cell standard deviation) and the number of participants. But it is easy to show that the t-test is a simple function of a correlation coefficient where one of the variables is dichotomous. Similarly, the F statistic of an analysis of variance is directly related to the correlation between the group means and a set of contrast coefficients. The use of meta-analysis to combine results from different studies has forced reEXPERIMENTAL APPROACHES 4 searchers to think about the size and consistency of their effects rather than the statistical significance of the effects. Indeed, realizing that r = √ F/(F + df) or √ t2/(t2 + df) (where df = degrees of freedom) did much to stop the complaint that personality coefficients of .3 were very small and accounted for less than 10% of the variance to be explained (Ozer, 2006). For suddenly, the highly significant F statistics reported for experimental manipulations in other subfields of psychology were shown to be accounting for even a smaller fraction of the variance of the dependent variable. The realization that statistics that seemed different are actually just transformations of each other forced experimentalists and correlationalists to focus on the inferences they can make from their data, rather the way in which the data are analyzed. The problems are what kind of inferences one can draw from a particular design, not whether correlations or experiments are the better way of studying the problem. That is, recognizing that correlations, regressions, t and F statistics are all special cases of the general linear model has allowed researchers to focus on the validity of the inferences drawn from the data, rather than on the seeming differences of experimental versus correlational statistics. Latent constructs, observed variables and the problems of inference Fundamental to the problem of inference is the distinction between the constructs that we think about and the variables that we measure and observe. This distinction between latent (unobserved) constructs and measured (observed) variables has been with us at least since Plato’s Allegory of the Cave in The Republic. Consider prisoners shackled in a cave and able to see only shadows (observed scores) on the cave wall of others (latent scores) walking past a fire. The prisoners attempt to make inferences about reality based upon what they can observe from the length and shape of the shadows. Individual differences in shadow length will correctly order individual differences in height, although real height can not be determined. To make this more complicated, as people approach the fire, their shadow lengths (the observed scores) will increase, even though their size (the latent score) has not changed. So it is for personality research. We are constrained to make inferences about latent variables based upon what we measure of observed variables. The problem may be shown diagrammatically (Figures 1 and 2) where boxes represent observed variables, circles latent constructs, and triangles experimental manipulations. Figure 1 is a simplified version of Figure 2, and shows how the relationship between an observed Person Variable and Outcome Variables (path A) when combined with an experimental manipulation (path B) and a potential observed interaction between the manipulation and the Person Variable (path C) reflect the interrelationships of latent variables as they are affected by an experimental manipulation (paths a, b, c, respectively) and attenuated by the reliability of measurement (r and s). Thus A=ras and B = bs and C = rcs. Our goal is to solve these equations for the unknowns (a,b,c, r, and s) in terms of the knowns (A, B, C). Figure 2 extends this analysis by adding in intervening Latent EXPERIMENTAL APPROACHES 5 Latent Person Trait Variable Experimental Manipulation Latent Outcome Variable Observed Person Variable Observed Outcome Variable c